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PREFACE  i 

The  work  described  in  this  publication  was  performed  by  the  , 

Mathematical  Analysis  Research  Corporation  (MARC)  under  contract  to 

* 

the  Jet  Propulsion  Laboratory,  an  operating  division  of  the  California 
Institute  of  Technology.  This  activity  is  sponsored  by  the  Jet 

i 

Propulsion  Laboratory  under  contract  NAS7-918,  RE182,  A187  with  the  i 

I 

National  Aeronautics  and  Space  Administration,  for  the  United  States  < 

Army  Intelligence  Center  and  School. 
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Mixing  Combination  and  the  Acceptance  Test 


I.  INTRODUCTION 
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'  When  all  of  the  appropriate  hypotheses  apply,  combinations  of  95% 
confidence  ellipses  produce  resultant  ellipses  which  are  themselves  95? 
ellipses.  In  other  reports  it  haJs  been  shown  that  when  one  or  more  of  the 
hypotheses  is  violated  (such  as  ysing  estimated  covariance  matrices  instead  of 
the  true  ones)  the  result  need  npt  be  a  95%  ellipse.  The  question  addressed 
in  this  report  is  how  applying  acceptance  tests  before  combination  affects  the 
confidence  level  of  Jhc  resultant  ellipse. 

The  discussions  in  this  oeibo  are  organized  as  follows! 

Section  II  -  A  model  of  the  way  combination  and  testing  mix; 

Section  III  -  A  discussion  of  results  and  conclusionsj 
Section  IV  -  Analysis  of  limit  cases  of  extreme  relative  sizej 
Section  V  -  Discussion  of  when  shape  can  and  cannot  be  Ignored, 
with  an  examplej  (Orientation  is  also  discussed.) 

Section  VI  -  Simulation  results  plus  a  discussion  of  the  meaning  of 
the  values  computed.  Some  readers  may  wish  to  skip  to 
this  section. 

In  a  later  report  the  Impact  of  the  number  of  such  combinations  performed  will 
be  analyzed.  Also  note  that  in  this  report  the  possibility  that  one  might 
attempt  to  combine  data  from  different  emitters  Is  not  addressed.  This  type 
of  error  could  also  have  a  significant  Impact  but  these  impacts  will  be 
addressed  in  a  later  report. 

II.  MODEL  OF  COMBINATION  AND  TESTING  MIXTURE  f }.  ■■«*- 

■  1  1  —  -  — — — —  —  ■  ,  7-  /  . 

For  this  report  it  Is  assumed  that  ^  • 

(1)  There  Is  truly  only  one  emitter.  I 

(2)  There  Is  no  initiation  of  new  ellipses  to  consider. 

(3)  The  acceptance  test  is  performed  at  the  95%  level 

(4)  If  the  acceptance  test  says  accept  then  the  incoming  ellipse  is 
combined  with  the  ellipse  already  In  the  data  base.  In  this  case 
the  resultant  ellipse  will  be  the  combined  ellipse. 

(5)  If  the  acceptance  test  says  reject  then  the  Incoming  ellipse  will 
be  dropped  and  the  original  ellipse  will  be  the  resultant  ellipse. 

Finally  note  that  in  this  report  we  only  discuss  the  results  of  bringing  in 
one  incoming  ellipse  one  time. 


III.  RESULTS  AND  CONCLUSIONS 

It  appears  that  confidence  levels  are  somewhat  lower  when  combination  and 
testing  are  mixed  than  they  would  otherwise  be. 

Theory  says  that  if  combination  is  done  without  testing,  the  resultant 
should  capture  the  true  emitter  95%  of  the  time.  From  this  we  inferred  that: 


If  passing  the  acceptance  test  implies  that 

«>  the  combined  ellipse  contains  the  true  (95*e)*  of  the  time 

Then  failing  the  acceptance  test  Implies  that 

■>  the  combined  ellipse  contains  the  true  (95-19e)*  of  the  time 

However,  failing  the  acceptance  test  really  implies  that 
->  the  combination  is  not  done 

->  the  original  ellipse  (failing  combination)  is  used  instead 
->  the  true  is  caught  X*  of  the  time  by  the  original  ellipse 

(Note  that  comparing  X*  versus  (95-1 9e)*  was  MARC's  first  concept  for 
studying  this  problem.) 

X*  depends  on  the  relative  size  of  the  ellipses  being  combined.  If  R 
denotes  the  ratio  of  the  major  axes  of  similarly  oriented  and  shaped  ellipses 
then 


Ratio  of 

percent  of  times  true  is 

New/Old 

caught  by  old  if  test  fails 

R 

x% 

Source 

(limit  case)  0 

0 % 

Analysis  ♦  simulation 

1 

about  70% 

Simulation 

(limit  case)  • 

95* 

Analysis  +  simulation 

The  impact  of  shape  can  only  be  inferred  in  a  general  way  at  this  time.  It 
appears,  however,  that  X%  for  nonsimilar  ellipses  could  be  based  on  ratio  if 
the  ambiguity  of  that  ratio  in  different  directions  is  tolerable. 

Unfortunately  it  is  not  possible  to  rotate  the  ellipse  to  reduce  the  range  of 
ratios.  For  example  differently  oriented  ellipses  of  the  same  size  and  shape 
produce  different  results  depending  on  orientation  which  in  general  are  not 
the  same  as  the  R-1  case  above.  (See  the  attached  examples  for  an  extreme 
case. ) 

The  natural  quantity  to  compare  with  X*  above  is  (95-1 9e)*.  MARC  was  not 
able  to  do  enough  simulations  to  determine  e  by  this  method  so  our  results  are 
based  on  analysis.  (Simulations  do  suggest  e  is  small  or  even  0.)  Analysis 
can  show  that  e»0  for  R-0  and  for  R-«>.  Furthermore,  symmetry  shows  that  R  and 
1 /R  would  produce  the  same  value  for  e. 

What  these  results  suggest  is  that  if  the  test  rejects  combination  of  a 
large  ellipse  and  a  small  ellipse  when  in  fact  they  do  come  from  the  same 
emitter  then  the  large  ellipse  probably  does  not  contain  the  emitter  but  the 
confidence  in  the  small  ellipse  is  not  impaired.  If  ellipses  are  comparable 
in  size  then  one's  confidence  in  both  ellipses  is  impaired  but  to  a  lesser 
extent.  Using  this  information  will  be  difficult,  however  as  other 
considerations  are  likely  to  come  into  play  in  practice. 


IV.  ANALYSIS  OF  EXTREME  CASES  IN  SIZE  RATIO  BETWEEN  THE  TWO  ELLIPSES 
A.  Background  Material 

The  analysis  performed  in  this  section  shall  use  results  from  MARC's 
report  entitled  "Testing  and  Combination  of  Confidence  Ellipses:  A  Geometric 
Analysis."  In  particular  in  that  report  it  was  established  that  there  is  an 
intersection  Interpretation  of  the  acceptance  test  where: 

1.  If  one  ellipse  contains  the  center  of  the  other  the  test  accepts. 


2.  If  the  ellipses  do  not  intersect  then  the  test  rejects. 

Note  there  are  unresolved  cases  left  by  these  two  criteria.  The  limit  cases 
we  will  study  have  the  property  that  the  probability  of  being  unresolved  by 
these  two  rules  goes  to  zero  in  the  limit  and  hence  can  be  ignored. 

8.  Analysis  of  Cases 

Case  1-  The  ellipse  already  in  the  data  base  is  very  much  smaller  than 
the  Incoming  ellipse. 

Fact  1 :  If  combined  then  the  resultant  ellipse  would  be  very 

similar  to  the  ellipse  already  in  the  data  base. 

Fact  2:  The  cases  of  interest  are  those  which  occur  with  some 

reasonable  frequency  (say  perhaps  within  the  99.99%  ellipse.) 

Fact  3:  For  cases  that  occur  with  any  reasonable  frequency  (as 

defined  above)  and  for  a  sufficiently  extreme  difference  in 
size  between  the  existing  and  incoming  ellipses,  then  the 
acceptance  test  accepts  in  approximately  the  same  cases  for 
which  the  incoming  ellipse  contains  the  emitter. 

The  Justification  may  be  formalized  (epsilon-delta  type 
arguments)  but  may  more  easily  be  thought  of  in  the  following 
way.  Graphs  are  provided  to  clarify  asertions  found  here, 
i)  In  the  limit  of  size  ratio  the  true  emitter  must  be 
relatively  close  to  the  small  ellipse  in  the  cases  of 
interest.  Technically  one  must  decide  how  many  digits 
of  precision  on  needs  then  go  out  to  the  ellipse  of  a 
correspondingly  high  confidence  level  (say  99.9999%) 
and  then  assert  a  sufficiently  high  percentage  cases 
of  interest  (depending  on  #  of  digits  of  accuracy 
needed)  are  accounted  for.  Although  it  means  going 
further  out  in  the  limit  variable  before  it  occurs  the 
99.9999?  ellipse  will  eventually  be  ’small’  relative 
to  the  larger  ellipse. 

ii)  The  test  accepts  in  approximately  the  same  cases  for 
which  the  incoming  ellipse  contains  the  emitter  and 
with  the  ellipse  sizes  so  different  intersection  means 
the  Incoming  ellipse  probably  will  contain  either  all 
or  none  of  existing  ellipse  and  surrounding  region. 
(Surrounding  region  meaning  the  '99.9999%  confidence 
region'  and  hence  would  include  the  true  in  cases  of 
interest. ) 

The  conclusion  that  results  from  this  analysis  is  if  combined  the  resultant 
ellipse  will  contain  the  true  emitter  location  95%  of  the  time  since  that 
ellipse  is  little  different  from  the  original  ellipse  in  the  data  base.  If 
test  fails,  however,  then  one  would  either  be  in 

1)  the  theoretically  very  rare  case  where  the  location  estimate  of  the 
original  ellipse  is  significantly  less  accurate  than  suggested  by  the 
confidence  ellipse 

or  ii)  the  case  where  the  incoming  ellipse  did  not  contain  the  true. 

In  other  words  5%  of  the  time  one  would  have  incoming  ellipses  not  containing 
the  true.  These  ellipses  might  go  back  into  the  data  base  for  possible 
classification  as  a  new  entity. 

These  conclusions  are  also  supported  by  the  pattern  of  simulations  in 
attachments  I  and  II. 

Case  2-The  incoming  ellipse  is  much  smaller  than  the  ellipse  already  In 
the  data  base. 

The  analysis  of  this  case  is  the  same  as  the  previous  case  with  roles 
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reversed.  The  significance  of  this  reversal  of  roles  is  that  the  acceptance 
test  will  pass  as  long  as  the  original  ellipse  contains  the  true  emitter 
location.  If  original  ellipse  doesn't  contain  the  true  emitter  location  then 
it  will  fail  to  combine.  The  larger  the  original  ellipse  the  longer  this 
problem  would  be  expected  to  persist. 

V.  SHAPE  AND  ORIENTATION 

The  extreme  case  analysis  of  the  impact  of  size  was  based  in  the  geometry 
of  testing  and  combination.  All  the  arguments  would  still  apply  if  the  shapes 
were  slightly  different.  Shape  only  matters  to  the  extent  that  it  makes  the 
ratio  of  sizes  ambiguous.  Orientation  also  affects  the  ambiguity  of  the  ratio 
of  sizes  as  can  be  seen  in  example  X.  In  fact  this  is  an  extreme,  but  still 
important  case.  The  ellipses  shown  are  of  equal  size  but  because  of 
orientation  the  ratio  of  sizes  is  amblgous.  The  results  shown  for  this  case 
are  typical. 

In  cases  where  the  major  axes  of  the  two  ellipses  point  in  the  same 
direction  and  in  cases  where  ellipses  are  nearly  round,  ratio  techniques  may 
be  used.  Note  that  two  ellipses  that  have  equal  axes  radii  but  different 
compass  orientations  are  different  rhapes  for  the  purposes  of  this  report. 

Also  note  that  differences  in  size  outweigh  the  impact  of  shape. 

VI.  SIMULATIONS 

In  reading  the  simulation  output  reports: 

Accepts  are  the  cases  where  the  acceptance  test  accepts  and  the 
combination  of  ellipses  is  performed.  The  number  catching  the 
true  in  this  case  is  the  number  of  combined  ellipses  catching 
the  true. 

Rejects  are  the  cases  where  the  acceptance  test  accepts  and  the 
combination  of  ellipses  is  not  performed.  The  number  catching 
the  true  in  this  case  is  the  number  of  times  the  ellipse  preexisting 
in  the  database  catches  the  true.  (The  other  ellipse  referred  to 
as  the  'incoming'  ellipse  has  no  impact  on  the  score.) 
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Case  Description:  Incoming  Ellipse  is  Much  Bigger  Than  The  Data  Base  Ellipse 
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Case  Description:  Incoming  Ellipse  is  Much  Smaller  Than  The  Data  Base  Ellipse 


Acceptance 
Test  Results 


Accepts 

Rejects 


Total  Number 
Of  Occurences 


Number  Catching 
The  True 


Percentage  Catching 
The  True 


0.9^65 

0.0323 
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Case  Description:  Approximately  Equal  Ellipse  Sizes 
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Case  Description:  Incoming  and  Base  Ellipse  Are  Orthogonal  and  ’Thin' 


Acceptance 
Test  Results 


Accepts 

Rejects 


Total  Number 
Of  Occurences 


Number  Catching 
The  True 


Percentage  Catching 
The  True  _ 


0.9*171 

0.5189 


